feat(nethttp-mysql): add net/http + MySQL sample for replay regression#224
Conversation
Adds a minimal net/http + database/sql + go-sql-driver/mysql sample that exercises the full MySQL connection and command phase end-to-end. Shape of the app: - /health issues db.Ping() (forces a real command roundtrip). - /users SELECTs from a seeded table. - /users/add INSERTs and returns the row. Why: Keploy's OSS pipeline consumes this sample as a regression guard for the MySQL replay command-phase read deadline. If the proxy ever re-introduces a zero or negative read deadline, the Go driver blocks inside db.Ping() during replay and the HTTP listener never binds :8080, making the regression loud in CI rather than silently hanging. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
There was a problem hiding this comment.
Pull request overview
Adds a new nethttp-mysql sample app (net/http + database/sql + go-sql-driver/mysql) intended to act as a CI regression guard for MySQL replay command-phase read-deadline issues by ensuring db.Ping() completes and the server successfully binds :8080.
Changes:
- Introduces a minimal HTTP server with
/health,/users, and/users/addbacked by a MySQLuserstable. - Adds module files (
go.mod/go.sum) for the new sample. - Adds Dockerfile + sample README with local-run and Keploy record/test instructions.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| nethttp-mysql/main.go | New MySQL-backed HTTP server + startup schema/seed logic used to exercise command-phase traffic. |
| nethttp-mysql/README.md | Local Docker + Keploy record/test instructions for the regression sample. |
| nethttp-mysql/Dockerfile | Container build/runtime packaging for the sample. |
| nethttp-mysql/go.mod | Declares the sample’s standalone Go module and MySQL driver dependency. |
| nethttp-mysql/go.sum | Locks dependency checksums for reproducible builds. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| for attempt := 1; attempt <= 30; attempt++ { | ||
| db, err = sql.Open("mysql", dsn) | ||
| if err == nil { | ||
| err = db.Ping() | ||
| if err == nil { | ||
| break | ||
| } | ||
| } | ||
| log.Printf("waiting for mysql (attempt %d): %v", attempt, err) | ||
| time.Sleep(2 * time.Second) | ||
| } | ||
| if err != nil { | ||
| log.Fatalf("could not connect to mysql after retries: %v", err) |
There was a problem hiding this comment.
In the retry loop, a new *sql.DB handle is created on every attempt, but failed attempts aren't closed. If Ping() fails, the previous handle can keep underlying connections/resources around. Consider calling Close() before reassigning, or Open() once and just retry Ping() on the same handle.
| for attempt := 1; attempt <= 30; attempt++ { | |
| db, err = sql.Open("mysql", dsn) | |
| if err == nil { | |
| err = db.Ping() | |
| if err == nil { | |
| break | |
| } | |
| } | |
| log.Printf("waiting for mysql (attempt %d): %v", attempt, err) | |
| time.Sleep(2 * time.Second) | |
| } | |
| if err != nil { | |
| log.Fatalf("could not connect to mysql after retries: %v", err) | |
| db, err = sql.Open("mysql", dsn) | |
| if err != nil { | |
| log.Fatalf("could not create mysql handle: %v; verify the DSN and MySQL driver configuration, then restart the service", err) | |
| } | |
| defer db.Close() | |
| for attempt := 1; attempt <= 30; attempt++ { | |
| err = db.Ping() | |
| if err == nil { | |
| break | |
| } | |
| log.Printf("waiting for mysql (attempt %d): %v; ensure the MySQL container is running and reachable at %s:%s, then retry", attempt, err, host, port) | |
| time.Sleep(2 * time.Second) | |
| } | |
| if err != nil { | |
| log.Fatalf("could not connect to mysql after retries: %v; confirm MySQL is healthy and the connection settings are correct, then restart the service", err) |
| } | ||
|
|
||
| var count int | ||
| _ = db.QueryRow("SELECT COUNT(*) FROM users").Scan(&count) |
There was a problem hiding this comment.
The result of QueryRow(...).Scan(&count) is intentionally ignored. If this query fails (e.g., missing permissions/table issues), count stays 0 and the app will try to seed, potentially masking the root cause. Please handle the Scan error and fail fast (or at least log it) so startup is deterministic.
| _ = db.QueryRow("SELECT COUNT(*) FROM users").Scan(&count) | |
| if err := db.QueryRow("SELECT COUNT(*) FROM users").Scan(&count); err != nil { | |
| log.Fatalf("failed to count existing users; verify the users table exists and the configured MySQL user has SELECT access: %v", err) | |
| } |
| } | ||
| w.Header().Set("Content-Type", "application/json") | ||
| _ = json.NewEncoder(w).Encode(users) | ||
| } |
There was a problem hiding this comment.
After iterating rows.Next(), rows.Err() is not checked, and JSON encoding errors are ignored. This can cause partial/empty responses to be returned as 200 OK even when iteration/encoding fails. Please check rows.Err() after the loop and handle/return any Encode error.
| http.Error(w, err.Error(), http.StatusInternalServerError) | ||
| return | ||
| } | ||
| id, _ := res.LastInsertId() |
There was a problem hiding this comment.
res.LastInsertId() error is ignored. If the driver doesn't support it or it fails, the handler will silently return id=0. Please handle the error and return a 500 (or similar) if the insert ID can't be retrieved.
| id, _ := res.LastInsertId() | |
| id, err := res.LastInsertId() | |
| if err != nil { | |
| http.Error(w, "failed to retrieve the inserted user ID; please retry or verify the database driver supports LastInsertId", http.StatusInternalServerError) | |
| return | |
| } |
| time.Sleep(2 * time.Second) | ||
| } | ||
| if err != nil { | ||
| log.Fatalf("could not connect to mysql after retries: %v", err) |
There was a problem hiding this comment.
This fatal error message doesn't give a clear next step for debugging (e.g., which env vars/host were used, or suggesting to check the MySQL container/network). Consider including the target host:port/db and a suggested action (verify MySQL is running/reachable and credentials are correct) to make CI/local failures faster to diagnose.
| log.Fatalf("could not connect to mysql after retries: %v", err) | |
| log.Fatalf("could not connect to MySQL after retries (target=%s:%s/%s): %v; verify MySQL is running and reachable at that address and that MYSQL_USER/MYSQL_PASSWORD/MYSQL_DATABASE are correct", host, port, dbname, err) |
| RUN go mod download || true | ||
| COPY . . | ||
| RUN go mod tidy && CGO_ENABLED=0 GOOS=linux go build -o app . |
There was a problem hiding this comment.
go mod download || true will hide real module download failures and can make builds succeed with missing deps until a later step fails. Also, running go mod tidy during image build makes the build non-deterministic and can mutate the module graph at build time. Prefer failing fast on go mod download and running go build without go mod tidy here (keep tidy as a repo-side step).
| RUN go mod download || true | |
| COPY . . | |
| RUN go mod tidy && CGO_ENABLED=0 GOOS=linux go build -o app . | |
| RUN go mod download | |
| COPY . . | |
| RUN CGO_ENABLED=0 GOOS=linux go build -o app . |
| keploy test -c "docker run -p 8080:8080 --name nethttp-mysql --network keploy-network \ | ||
| -e MYSQL_HOST=mysql -e MYSQL_USER=root -e MYSQL_PASSWORD=password \ | ||
| -e MYSQL_DATABASE=testdb nethttp-mysql" \ | ||
| --containerName nethttp-mysql --network-name keploy-network \ |
There was a problem hiding this comment.
The Keploy CLI flag here uses --containerName, but other samples in this repo consistently use --container-name (e.g., http-postgres/README.md:58, fasthttp-postgres/README.md:25). If --containerName isn't a valid alias, the command will fail. Please align this to --container-name for consistency and correctness.
| --containerName nethttp-mysql --network-name keploy-network \ | |
| --container-name nethttp-mysql --network-name keploy-network \ |
| } | ||
|
|
||
| func getUsers(w http.ResponseWriter, r *http.Request) { | ||
| rows, err := db.Query("SELECT id, name, email FROM users") |
There was a problem hiding this comment.
The /users query has no ORDER BY, so row order is not guaranteed and can change as the table grows/changes. For a sample used in record/replay regression, this can introduce flaky diffs. Consider adding an explicit ordering (e.g., by id) to keep responses deterministic.
| rows, err := db.Query("SELECT id, name, email FROM users") | |
| rows, err := db.Query("SELECT id, name, email FROM users ORDER BY id") |
The e2e pipeline cannot discriminate a fix from a regression for this specific bug. Triggering the zero/overflow read-deadline path requires SQLDelay = 0 (or a real seconds-valued Duration) in the replay command-phase loop. SQLDelay is derived exclusively from Test.Delay via `time.Duration(Test.Delay) * time.Second`, and the same value gates keploy's app-startup wait before replay begins. Pushing Test.Delay = 0 into the driver script to exercise the zero-deadline path also causes keploy to replay before the sample binds :8080, so the matrix fails for the wrong reason on both buggy and fixed binaries. With any non-zero Test.Delay the two pre-fix unit bugs (`time.Duration(Test.Delay)` without `* time.Second` in the caller and `2 * time.Second * time.Duration(opts.SQLDelay)` in the proxy) cancel out and the buggy released binary computes a valid positive deadline. Net: all three matrix cells would stay green regardless of whether the fix is present, so the entry provides no signal. Drop the matrix entry, the driver script, and the prepare_and_run trigger change that was only there to let this stacked branch run outside main. The fix remains guarded by the derivation unit test in query_readtimeout_test.go, which pins the math across the zero, sub-second, seconds, and hours cases. The nethttp-mysql sample in keploy/samples-go#224 is unaffected and stays in place as a general MySQL coverage sample. - Remove nethttp-mysql entry from .github/workflows/golang_docker.yml. - Delete .github/workflows/test_workflow_scripts/golang/nethttp_mysql/ golang-docker.sh. - Restore the main-only branch filter on pull_request and push in .github/workflows/prepare_and_run.yml. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
The e2e pipeline cannot discriminate a fix from a regression for this specific bug. Triggering the zero/overflow read-deadline path requires SQLDelay = 0 (or a real seconds-valued Duration) in the replay command-phase loop. SQLDelay is derived exclusively from Test.Delay via `time.Duration(Test.Delay) * time.Second`, and the same value gates keploy's app-startup wait before replay begins. Pushing Test.Delay = 0 into the driver script to exercise the zero-deadline path also causes keploy to replay before the sample binds :8080, so the matrix fails for the wrong reason on both buggy and fixed binaries. With any non-zero Test.Delay the two pre-fix unit bugs (`time.Duration(Test.Delay)` without `* time.Second` in the caller and `2 * time.Second * time.Duration(opts.SQLDelay)` in the proxy) cancel out and the buggy released binary computes a valid positive deadline. Net: all three matrix cells would stay green regardless of whether the fix is present, so the entry provides no signal. Drop the matrix entry, the driver script, and the prepare_and_run trigger change that was only there to let this stacked branch run outside main. The fix remains guarded by the derivation unit test in query_readtimeout_test.go, which pins the math across the zero, sub-second, seconds, and hours cases. The nethttp-mysql sample in keploy/samples-go#224 is unaffected and stays in place as a general MySQL coverage sample. - Remove nethttp-mysql entry from .github/workflows/golang_docker.yml. - Delete .github/workflows/test_workflow_scripts/golang/nethttp_mysql/ golang-docker.sh. - Restore the main-only branch filter on pull_request and push in .github/workflows/prepare_and_run.yml. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
…se (#4095) * fix(mysql): prevent zero/overflow read deadline in replay command phase The command-phase loop in pkg/agent/proxy/integrations/mysql/replayer/ query.go derived its per-iteration read deadline as readTimeout := 2 * time.Second * time.Duration(opts.SQLDelay) which was wrong in two directions: - When the HTTP caller sent SQLDelay=0 (a legitimate shape once the field is serialized across the agent/client boundary), the product was 0s. SetReadDeadline(time.Now().Add(0)) set an already-expired deadline, ReadPacketBuffer returned a timeout without reading any bytes, and the timeout branch's 50ms backoff became a hot loop that never consumed the client's MySQL commands. The client driver sat in a blocking Read waiting for a response that would never come. - When SQLDelay was a genuine seconds-valued time.Duration (e.g. 5s = 5e9), the product 2 * 1e9 * 5e9 = 1e19 overflowed int64, wrapped negative, and produced the same expired-deadline hot loop from the opposite end. The derivation is replaced with readTimeout := 2 * opts.SQLDelay if readTimeout < time.Second { readTimeout = 2 * time.Second } which treats SQLDelay as a duration (matching how the rest of the codebase uses time.Duration) and guarantees a safe positive deadline even when the caller sends zero. The two callers that build OutgoingOptions (pkg/service/replay and pkg/service/runner) were also feeding r.config.Test.Delay (a uint64 meaning seconds) through time.Duration() without multiplying by time.Second, producing a nanosecond-valued Duration. Those are corrected to `time.Duration(Delay) * time.Second` so the knob's unit matches user intent and the startup-wait path (which already uses `* time.Second`). Regression guard: a new golang_docker matrix entry (nethttp-mysql) drives a net/http + database/sql sample end-to-end. If this bug ever re-lands, the sample's db.Ping() will block during replay, the HTTP server will not bind :8080, and every replayed test case will fail with connection reset — making the regression loud in CI. A small unit test in query_readtimeout_test.go pins the derivation math itself so future refactors cannot silently reintroduce a zero or sub-second deadline. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com> * ci: remove main-only branch filter from prepare_and_run triggers - Drops `branches: [main]` from both the pull_request and push triggers in prepare_and_run.yml so the added dns_strict_resolver and nethttp-mysql pipelines (plus the rest of the umbrella) run on PRs and pushes targeting any base branch, not just main. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com> * chore: retrigger CI pipeline - Empty commit to retrigger the CI pipeline. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com> * ci: drop nethttp-mysql e2e wiring for this fix The e2e pipeline cannot discriminate a fix from a regression for this specific bug. Triggering the zero/overflow read-deadline path requires SQLDelay = 0 (or a real seconds-valued Duration) in the replay command-phase loop. SQLDelay is derived exclusively from Test.Delay via `time.Duration(Test.Delay) * time.Second`, and the same value gates keploy's app-startup wait before replay begins. Pushing Test.Delay = 0 into the driver script to exercise the zero-deadline path also causes keploy to replay before the sample binds :8080, so the matrix fails for the wrong reason on both buggy and fixed binaries. With any non-zero Test.Delay the two pre-fix unit bugs (`time.Duration(Test.Delay)` without `* time.Second` in the caller and `2 * time.Second * time.Duration(opts.SQLDelay)` in the proxy) cancel out and the buggy released binary computes a valid positive deadline. Net: all three matrix cells would stay green regardless of whether the fix is present, so the entry provides no signal. Drop the matrix entry, the driver script, and the prepare_and_run trigger change that was only there to let this stacked branch run outside main. The fix remains guarded by the derivation unit test in query_readtimeout_test.go, which pins the math across the zero, sub-second, seconds, and hours cases. The nethttp-mysql sample in keploy/samples-go#224 is unaffected and stays in place as a general MySQL coverage sample. - Remove nethttp-mysql entry from .github/workflows/golang_docker.yml. - Delete .github/workflows/test_workflow_scripts/golang/nethttp_mysql/ golang-docker.sh. - Restore the main-only branch filter on pull_request and push in .github/workflows/prepare_and_run.yml. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com> * chore: retrigger CI pipeline - Empty commit to kick off the workflow after rebasing the branch onto main. Signed-off-by: Asish Kumar <officialasishkumar@gmail.com> --------- Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
Describe the changes that are made
Adds a minimal
nethttp-mysqlsample — standard librarynet/httpplus
database/sqlwithgo-sql-driver/mysql— that exercises thefull MySQL connection and command phase end-to-end:
/healthcallsdb.Ping()(forces a real command roundtrip)./usersSELECTs from a seededuserstable./users/addINSERTs and returns the new row.Why this sample exists: Keploy's OSS pipeline will consume it as a
regression guard for the MySQL replay command-phase read deadline.
If the proxy ever re-introduces a zero or negative deadline, the Go
driver blocks inside
db.Ping()during replay, the HTTP listenernever binds
:8080, and every replayed request fails withconnection reset by peer— making the regression loud in CIrather than silently hanging on the user's side.
The companion PR in keploy/keploy (fix: prevent zero/overflow read
deadline in MySQL replay command phase) adds the pipeline script that
drives this sample and the fix for the proxy math.
Links & References
Closes: keploy/keploy#4102
🔗 Related PRs
🐞 Related Issues
📄 Related Documents
What type of PR is this? (check all applicable)
Added e2e test pipeline?
golang_dockermatrix entrynethttp-mysqland the driver scriptat
.github/workflows/test_workflow_scripts/golang/nethttp_mysql/golang-docker.sh.Added comments for hard-to-understand areas?
main.goexplainswhy the app is shaped this way (regression guard).
Added to documentation?
keploy record/test instructions.
Are there any sample code or steps to test the changes?
Local smoke test:
Self Review done?
Any relevant screenshots, recordings or logs?
🧠 PR Semantics
feat(nethttp-mysql): add net/http + MySQL sample for replay regressionfeat/nethttp-mysql-hotloop-regression